Shared Fixture
The book has now been published and the content of this chapter has likely changed substanstially.Please see page 317 of xUnit Test Patterns for the latest information.
Also known as: Shared Context, Leftover Fixture, Reused Fixture, Stale Fixture
How can we avoid Slow Tests?
What fixture strategy should we use?
We reuse the same instance of the test fixture across many tests.
Sketch Shared Fixture embedded from Shared Fixture.gif
To execute an automated test, we require a text fixture that is well understood and completely deterministic. Setting up a Fresh Fixture (page X) can be time consuming, especially when dealing with complex system state stored in a test database.
We make our tests run faster by reusing the same fixture for several or many tests?
How It Works
The concept is pretty simple: we create a Standard Fixture (page X) fixture that outlasts the lifetime of a single Testcase Object (page X). This allows multiple tests to reuse the same test fixture without destroying it and recreating it between tests. A Shared Fixture can either be a Prebuilt Fixture that is reused by one or more tests in many test runs, or a fixture created by one test and reused by another test within the same test run. In either case, the key thing is that many tests do not create their own fixture but reuse one "left over" from some other activity. This makes the tests run faster because they have less fixture set up to do and that may result in the test automater having to do less work to define the fixture for each test.
When To Use It
Regardless of why we use them, Shared Fixtures come with some baggage that it is better to understand before we head down this path. The biggest issue with a Shared Fixture is that it can lead to "collisions" between tests possibly resulting in Erratic Tests (page X) since tests may depend on the outcomes of other tests. Another issue is that a fixture designed to serve many tests is bound to be much more complicated than the Minimal Fixture (page X) needed for a single test. This will typically take more effort to design and can lead to a Fragile Fixture (see Fragile Test on page X) down the road when you need to modify it.
A Shared Fixture will often result in a Obscure Test (page X) because the fixture is not constructed inside the test. This can be mitigated by the use of Finder Methods (see Test Utility Method on page X) with Intent Revealing Names[SBPP] to access the relevant parts of the fixture.
There are two valid reasons for using a Shared Fixture and some misguided ones. Many of the variations have been devised primarily to mitigate the negative consequences of using a Shared Fixture. So, what are good reasons for using a Shared Fixture?
Variation: Slow Tests
We can use a Shared Fixture when we cannot afford to build a new Fresh Fixture for each test. Typically, this will be because it takes too much processing to build a new fixture for each test and that often leads to Slow Tests (page X). This most commonly occurs when testing with real test databases due to the high cost of creating each of the records. This tends to be exacerbated by using the API of the system under test (SUT) to create the reference data since the SUT often does a lot of input validation which may involve reading some of the just-written records.
A better solution is to make the tests run faster by not interacting with
the database at all. For a more complete list of options, see the solutions
to Slow Tests and the sidebar Faster Tests Without Shared Fixtures (page X)
Include the sidebar 'Faster Tests Without Shared Fixtures' on opposite page.
.
Variation: Incremental Tests
Another use of Shared Fixtures is when we have a long complex sequence of actions each of which depends on the previous actions. In customer tests this may show up as a workflow while in unit tests it may be a sequence of method calls on the same object. This might be tested using a single Eager Test (see Assertion Roulette on page X). The alternative is to put each distinct action into a separate Test Method (page X) that builds upon the actions of a previous test operating on a Shared Fixture. This is an example of Chained Tests (page X).This is how testers in the "testing" (or "QA") community often operate. They set up a fixture and then run a sequence of tests that each build upon the fixture. They do have one significant advantage over our Fully Automated Tests (see Goals of Test Automation on page X): when a test part way through the chain fails, they are available to make decisions about how to recover or whether it is worth proceeding at all. Our automated tests just keep running and running and running and many of them will generate test failures or errors because the fixture was not as expected so the SUT behaved (probably correctly) differently. The resulting test results can obscure the real cause of the failure in a sea of red. With some experience it is often possible to recognize the failure pattern and deduce the root cause.(It may not be as simple as looking at the first test that failed.)
This trouble-shooting can be made simpler by starting each Test Method with one or more Guard Assertions (page X) that document the assumptions the Test Method makes about the state of the fixture. When these assertions fail, they tell us to look elsewhere; either at tests failing earlier in the test suite or at the order the tests were run in.
Implementation Notes
A key implementation consideration with Shared Fixtures is how do tests know about the objects in the Shared Fixture so they can (re)use them? Since the point of Shared Fixture is to save execution time and effort by having multiple tests use the same instance of the test fixture, we'll need to keep a reference to the fixture we create so that we can find the fixture if it already exists and inform other tests that it now exists once we have constructed it. We have more choices available to us with Per-Run Fixtures because we can "remember" the fixture we set up in code more easily than a Prebuilt Fixture (page X) set up by a different program. We could just hard-code the identifiers (e.g. database keys) of the fixture objects into all our tests but that would result in a Fragile Fixture. To avoid this, we need to keep a reference to the fixture when we create it and we need to make it possible for all the tests to access the reference.
Variation: Per-Run Fixture
The simplest form of Shared Fixture is the Per-Run Fixture where we set up the fixture at the beginning of a test run to be shared by the tests within the run. Ideally, the fixture won't outlive the test run and we don't have to worry about interactions between test runs such a Unrepeatable Tests (a cause of Erratic Tests.) If the fixture is persistence, such as when it is stored in a database, we may need to do explicit fixture tear down.
If a Per-Run Fixture is only being shared within a single Testcase Class (page X), the simplest solution is to use a class variable for each fixture object we need to hold a reference to and then use either Lazy Setup (page X) or SuiteFixture Setup (page X) to initialize them just before the first test in the suite is run. If we want to share the test fixture between many Testcase Classes we'll need to use a Setup Decorator (page X) to hold the setUp and tearDown methods and a Test Fixture Registry (see Test Helper on page X) (which could just be the test database) to access the fixture.
Variation: Immutable Shared Fixture
The problem with Shared Fixtures is that they lead to Erratic Tests if tests modify the Shared Fixture (page X). This is because Shared Fixtures violate the principle of Independent Test (see Principles of Test Automation on page X). We can avoid this by making the Shared Fixture immutable; we partition the fixture needed by tests into two logical parts. The first part is the stuff that every test needs to have present but which is never modified by any tests. This is the Immutable Shared Fixture. The second part is the objects which any test needs to modify or delete. These objects should be built by each test as a Fresh Fixture.
The hardest part of applying Immutable Shared Fixture is deciding what constitutes a change to an object. The key guideline is that if any test perceives something done by another test as a change to an object in the Immutable Shared Fixture, then it shouldn't be allowed in any test with which it shares the fixture. Most commonly, the Immutable Shared Fixture consists of reference data that is needed by the actual per-test fixtures. The per-test fixtures can then be built as a Fresh Fixture on top of the Immutable Shared Fixture.
Motivating Example
This example shows a Testcase Class which is setting up the test fixture using Implicit Setup (page X). Each Test Method uses an instance variable to access the contents of the fixture.
public void testGetFlightsByFromAirport_OneOutboundFlight() throws Exception {
setupStandardAirportsAndFlights();
FlightDto outboundFlight = findOneOutboundFlight();
// Exercise System
List flightsAtOrigin = facade.getFlightsByOriginAirport(
outboundFlight.getOriginAirportId());
// Verify Outcome
assertOnly1FlightInDtoList( "Flights at origin", outboundFlight,
flightsAtOrigin);
}
public void testGetFlightsByFromAirport_TwoOutboundFlights() throws Exception {
setupStandardAirportsAndFlights();
FlightDto[] outboundFlights = findTwoOutboundFlightsFromOneAirport();
// Exercise System
List flightsAtOrigin = facade.getFlightsByOriginAirport(
outboundFlights[0].getOriginAirportId());
// Verify Outcome
assertExactly2FlightsInDtoList( "Flights at origin", outboundFlights,
flightsAtOrigin);
}
Example StandardTestFixture embedded from java/com/clrstream/ex6/services/test/FlightManagementFacadeTest.java
Note that the setUp method is run once for each Test Method. If the fixture setup is fairly complex and involves accessing a database, this could result in Slow Testss.
Refactoring Notes
To convert a Testcase Class from a Standard Fixture to a Shared Fixture, we simply convert the instance variables into class variables to make the fixture outlast the creating Testcase Object. We then need to initialize the class variables just once to avoid recreating it for each Test Method; Lazy Setup is an easy way to do this. Of course, there are other ways to set up the Shared Fixture such as Setup Decorator or SuiteFixture Setup.
Example: Shared Fixture
This example shows the fixture converted to a Shared Fixture set up using Lazy Setup.
protected void setUp() throws Exception {
if (sharedFixtureInitialized) {
return;
}
facade = new FlightMgmtFacadeImpl();
setupStandardAirportsAndFlights();
sharedFixtureInitialized = true;
}
protected void tearDown() throws Exception {
// Cannot delete any objects because we don't know
// whether or not this is the last test
}
Example LazyFixtureInitialization embedded from java/com/clrstream/ex6/services/test/SharedFixtureFlightManagementFacadeTest.java
The Lazy Initialization[SBPP] logic in the setUp method ensures that the Shared Fixture is created whenever the class variable is uninitialized. The Test Methods have also been modified to use a Finder Method to access the contents of the fixture:
public void testGetFlightsByFromAirport_OneOutboundFlight() throws Exception {
FlightDto outboundFlight = findOneOutboundFlight();
// Exercise System
List flightsAtOrigin = facade.getFlightsByOriginAirport(
outboundFlight.getOriginAirportId());
// Verify Outcome
assertOnly1FlightInDtoList( "Flights at origin", outboundFlight,
flightsAtOrigin);
}
public void testGetFlightsByFromAirport_TwoOutboundFlights() throws Exception {
FlightDto[] outboundFlights = findTwoOutboundFlightsFromOneAirport();
// Exercise System
List flightsAtOrigin = facade.getFlightsByOriginAirport(
outboundFlights[0].getOriginAirportId());
// Verify Outcome
assertExactly2FlightsInDtoList( "Flights at origin", outboundFlights,
flightsAtOrigin);
}
Example SharedTestFixture embedded from java/com/clrstream/ex6/services/test/SharedFixtureFlightManagementFacadeTest.java
I haven't shown the details of how the Test Utility Methods such as setupStandardAirportsAndFlights are implemented since it isn't very important to understanding this example. It should be enough to understand that it creates the airports and flights and stores references to them in static variables so that all the Test Method can access the same fixture.
Example: Immutable Shared Fixture
Here's an example of Shared Fixture "pollution":
public void testCancel_proposed_p()throws Exception {
// shared fixture
BigDecimal proposedFlightId = findProposedFlight();
// exercise SUT
facade.cancelFlight(proposedFlightId);
// verify outcome:
try{
assertEquals(FlightState.CANCELLED, facade.findFlightById(proposedFlightId));
} finally {
// tearDown:
// try to undo the damage; hope this works!
facade.overrideStatus( proposedFlightId, FlightState.PROPOSED);
}
}
Example SharedFixturePollution embedded from java/com/clrstream/ex6/services/test/SharedFixtureFlightManagementFacadeTest.java
We can avoid this by making the Shared Fixture immutable; we partition the fixture needed by tests into two logical parts. The first part is the stuff that every test needs to have present but which is never modified by any tests. This is the Immutable Shared Fixture. The second part is the objects which any test needs to modify or delete. These objects should be built by each test as a Fresh Fixture.
Here's the same test modified to use an Immutable Shared Fixture. We simply created our own mutableFlight within the test.
public void testCancel_proposed() throws Exception {
// fixture setup:
BigDecimal mutableFlightId = createFlightBetweenInsigificantAirports();
// exercise SUT
facade.cancelFlight(mutableFlightId);
// verify outcome:
assertEquals( FlightState.CANCELLED, facade.findFlightById(mutableFlightId));
// tearDown:
// None required since were are letting the SUT create
// new IDs for each flight. We might need to clean out
// the database eventually.}
Example ImmutableSharedFixture embedded from java/com/clrstream/ex6/services/test/SharedFixtureFlightManagementFacadeTest.java
Note that we don't need any fixture tear down logic in this version of the test because we are letting the SUT use a Distinct Generated Value (see Generated Value on page X) by not supplying a flight number. And we are using the predefined dummyAirport1 and dummyAirport2 to avoid changing the number of flights on airports used by other tests. Therefore, the mutable flights can accumulate in the database without causing us any trouble.
Copyright © 2003-2008 Gerard Meszaros all rights reserved